Multi-touch Detection and Semantic Response on Non-parametric Rear-projection Surfaces

The ability of human beings to physically touch our surroundings has had a profound impact on our daily lives. Young children learn to explore their world by touch; likewise, many simulation and training applications benefit from natural touch interactivity. As a result, modern interfaces supporting touch input are ubiquitous. Typically, such interfaces are implemented on integrated touch-display surfaces with simple geometry that can be mathematically parameterized, such as planar surfaces and spheres; for more complicated non-parametric surfaces, such parameterizations are not available. In this dissertation, we introduce a method for generalizable optical multi-touch detection and semantic response on uninstrumented non-parametric rear-projection surfaces using an infrared-light-based multi-camera multi-projector platform. In this paradigm, touch input allows users to manipulate complex virtual 3D content that is registered to and displayed on a physical 3D object. Detected touches trigger responses with specific semantic meaning in the context of the virtual content, such as animations or audio responses. The broad problem of touch detection and response can be decomposed into three major components: determining if a touch has occurred, determining where a detected touch has occurred, and determining how to respond to a detected touch. Our fundamental contribution is the design and implementation of a relational lookup table architecture that addresses these challenges through the encoding of coordinate relationships among the cameras, the projectors, the physical surface, and the virtual content. Detecting the presence of touch input primarily involves distinguishing between touches (actual contact events) and hovers (near-contact proximity events). We present and evaluate two algorithms for touch detection and localization utilizing the lookup table architecture. One of the algorithms, a bounded plane sweep, is additionally able to estimate hover-surface distances, which we explore for interactions above surfaces. The proposed method is designed to operate with low latency and to be generalizable. We demonstrate touch-based interactions on several physical parametric and non-parametric surfaces, and we evaluate both system accuracy and the accuracy of typical users in touching desired targets on these surfaces. In a formative human-subject study, we examine how touch interactions are used in the context of healthcare and present an exploratory application of this method in patient simulation. A second study highlights the advantages of touch input on content-matched physical surfaces achieved by the proposed approach, such as decreases in induced cognitive load, increases in system usability, and increases in user touch performance. In this experiment, novice users were nearly as accurate when touching targets on a 3D head-shaped surface as when touching targets on a flat surface, and their self-perception of their accuracy was higher

Artificial intelligence approaches to predict coronary stenosis severity using non-invasive fractional flow reserve

Fractional flow reserve is the current reference standard in the assessment of the functional impact of a stenosis in coronary heart disease. In this study, three models of artificial intelligence of varying degrees of complexity were compared to fractional flow reserve measurements. The three models are the multivariate polynomial regression, which is a statistical method used primarily for correlation; the feed-forward neural network; and the long short-term memory, which is a type of recurrent neural network that is suited to modelling sequences. The models were initially trained using a virtual patient database that was generated from a validated one-dimensional physics-based model. The feed-forward neural network performed the best for all test cases considered, which were a single vessel case from a virtual patient database, a multi-vessel network from a virtual patient database, and 25 clinically invasive fractional flow reserve measurements from real patients. The feed-forward neural network model achieved around 99% diagnostic accuracy in both tests involving virtual patients, and a respectable 72% diagnostic accuracy when compared to the invasive fractional flow reserve measurements. The multivariate polynomial regression model performed well in the single vessel case, but struggled on network cases as the variation of input features was much larger. The long short-term memory performed well for the single vessel cases, but tended to have a bias towards a positive fractional flow reserve prediction for the virtual multi-vessel case, and for the patient cases. Overall, the feed-forward neural network shows promise in successfully predicting fractional flow reserve in real patients, and could be a viable option if trained using a large enough data set of real patients

Optical Touch Sensing On Non-Parametric Rear-Projection Surfaces

The field of augmented reality (AR) has introduced many novel input and output approaches for human-computer interaction. As touching physical objects with the fingers or hands is both natural and intuitive, touch-based graphical interfaces are ubiquitous, but many such interfaces are limited to flat screens or simple objects. We propose an optical method for multi-Touch detection and response on non-parametric surfaces with dynamic rear-projected imagery, which we demonstrate on two head-shaped surfaces. We are interested in exploring the advantages of this approach over two-dimensional touch input displays, particularly in healthcare training scenarios

Cognitive And Touch Performance Effects Of Mismatched 3D Physical And Visual Perceptions

While research in the field of augmented reality (AR) has produced many innovative human-computer interaction techniques, some may produce physical and visual perceptions with unforeseen negative impacts on user performance. In a controlled human-subject study we investigated the effects of mismatched physical and visual perception on cognitive load and performance in an AR touching task by varying the physical fidelity (matching vs. non-matching physical shape) and visual mechanism (projector-based vs. HMD-based AR) of the representation. Participants touched visual targets on four corresponding physical-visual representations of a human head. We evaluated their performance in terms of touch accuracy, response time, and a cognitive load task requiring target size estimations during a concurrent (secondary) counting task. After each condition, participants completed questionnaires concerning mental, physical, and temporal demands; stress; frustration; and usability. Results indicated higher performance, lower cognitive load, and increased usability when participants touched a matching physical head-shaped surface and when visuals were provided by a projector from underneath

Ucf @ Trecvid 2009: High-Level Feature Extraction

This year, the University of Central Florida participated in the high level feature extraction task (HLF). The goal of high level feature extraction is to identify in videos specific shots that contain concepts such as \bus, \person playing soccer, and \boat/ship. In our submissions, we focused on addressing the large imbalance between the positive and negative training examples. Specifically, we implemented a method called bootstrapping that identifies the best subset of negative examples to train on. In our experiments, we found bootstrapping significantly lowered the probability of false alarm while also improving the probability of detection. Additionally, we also explored different word weighting techniques. In the bag of words approach, certain words may be more discriminative than others; these words should be weighted more. This task served as a project for several students participating in the Research Experience for Undergraduates program (REU) at UCF

Touch Sensing On Non-Parametric Rear-Projection Surfaces: A Physical-Virtual Head For Hands-On Healthcare Training

We demonstrate a generalizable method for unified multitouch detection and response on a human head-shaped surface with a rear-projection animated 3D face. The method helps achieve hands-on touch-sensitive training with dynamic physical-virtual patient behavior. The method, which is generalizable to other non-parametric rear-projection surfaces, requires one or more infrared (IR) cameras, one or more projectors, IR light sources, and a rear-projection surface. IR light reflected off of human fingers is captured by cameras with matched IR pass filters, allowing for the localization of multiple finger touch events. These events are tightly coupled with the rendering system to produce auditory and visual responses on the animated face displayed using the projector(s), resulting in a responsive, interactive experience. We illustrate the applicability of our physical prototype in a medical training scenario

Optical Touch Sensing On Nonparametric Rear-Projection Surfaces For Interactive Physical-Virtual Experiences

We demonstrate a generalizable method for unified multitouch detection and response on various nonparametric and parametric surfaces to support interactive physical-virtual experiences. The method employs multiple infrared (IR) cameras, one or more projectors, IR light sources, and a rear-projection surface. IR light reflected off human fingers is captured by cameras with matched IR pass filters, allowing for the detection and localization of multiple simultaneous finger-touch events. The processing of these events is tightly coupled with the rendering system to produce auditory and visual responses displayed on the surface using the projector(s) to achieve a responsive, interactive, physical-virtual experience. We demonstrate the method on two nonparametric face-shaped surfaces and a planar surface. We also illustrate the approach’s applicability in an interactive medical training scenario using one of the head surfaces to support hands-on, touch-sensitive medical training with dynamic physical-virtual patient behavior

University Of Central Florida At Trecvid 2008 Content Based Copy Detection And Surveillance Event Detection

In this paper, we describe our approaches and experiments in content-based copy detection (CBCD) and surveillance event detection pilot (SEDP) tasks of TRECVID 2008. We have participated in the video-only CBCD task and four of the SEDP events. The CBCD method relies on sequences of invariant global image features and efficiently matching and ranking of those sequences. The normalized Hu-moments are proven to be invariant to many transformations, as well as certain level of noise, and thus are the basis of our system. The most crucial property of proposed CBCD system is that it relies on the sequence matching rather than independent frame correspondences. The experiments have shown that this approach is quite useful for matching videos under extensive and strong transformations which make single frame matching a challenging task. This methodology is proven to be fast and produce high F1 detection scores in the TRECVID 2008 task evaluation. We also submitted four individual surveillance event detection systems. Person-Runs , Object-Put , Opposing-Flow and Take-Picture are the four selected events. The systems rely on low level vision properties such as optical flow and image intensity as well as heuristics based on a given event and context

Exploring Album Structure For Face Recognition In Online Social Networks

In this paper, we propose an album-oriented face-recognition model that exploits the album structure for face recognition in online social networks. Albums, usually associated with pictures of a small group of people at a certain event or occasion, provide vital information that can be used to effectively reduce the possible list of candidate labels. We show how this intuition can be formalized into a model that expresses a prior on how albums tend to have many pictures of a small number of people. We also show how it can be extended to include other information available in a social network. Using two real-world datasets independently drawn from Facebook, we show that this model is broadly applicable and can significantly improve recognition rates. © 2014 Elsevier B.V. All rights reserved